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II.  PRELIMINARIES 


Let  R  be  the  real  line,  and  let  B  be  the  usual  Borel  a-field  on  R.  Let  R  be  the  two-sided 
sequence  space,  and  let  B  be  the  Borel  e-field  on  R  that  is  generated  by  the  product  topology 
on  R  .  We  consider  a  real-valued  discrete-time  process,  {Xn,  °°<n<°°),  whose  measure  po  is 
known  and  is  defined  on  B  .  We  name  {xn,  -°°<n<«>)  the  nominal  process  ,  and  we  denote  by 
(xn,  -oo<n<oo)  data  realizations  generated  by  it.  Let  xn  =  xn(x)n  ')  denote  the  optimal  one-step 

n-i  A 

mean-squared  prediction  operation,  given  the  sequence  realization  Xj  (xr  l</<n-l),  when 
(xn,  —»<no°}  is  generated  by  the  nominal  process.  Then,  if  gn  =  gn(x]n  ')  denotes  some  scalar 


real-valued  function  on  the  sequence  xt  ,  we  have: 


C>o’  V  =  inrC>o’g„) 


vx^-v^r1} 


;  where  E  {  )  denotes  expectation  with  respect  to  the  measure  u  ,  where 

0 


n  ^ 

Xj  =  {Xr  l</<n},  and  where. 


c  (jj.  ,  g  )  E  {[X  -g  (X"  ')]2J 

nvro  °rr  u  11  n  1  /J  * 

—  ^ o 


The  expression  in  (3)  is  called  the  one-step  prediction  error  induced  by  gn  at  |io.  Let  L 
denote  the  class  of  all  the  scalar  real-valued  linear  functions  defined  on  Rn.  Let  then 

L  I.  „-l 

x  =  x  (x,  )  be  such  that: 

n  n  x  1  7 


3 


t-  i. 

c  (u  ,  x  )  =  inf  c  (u.  ,  g  } 

n  'll  n  '  n'o  &n  ' 


i  1  '.*  •  *  (/  *,*  (..•  %,*  I.1  v  *r . , ‘r . *»  .V  *r  - V  '*  .*>  .*>  >»♦  -*  -  1 


L 

Then,  xn  is  called  die  optimal  linear  one-step  mean  squared  predictor  at  po,  given  the 
sequence  realization  x"  ',  and  generally. 


c  (p  ,  *  )  <  e  (p  .  X  ) 
nvro’  n 7  nVfo  n ' 


n-l  ^  I-  n-1 

If  the  measure  po  is  Gaussian,  then  xn(\t  )  &n  (Xj  ),  -Vn,  and  (5)  is  then  satisfied  with 


equality  for  all  n.  If  (,to  is  non  Gaussian,  then  (5)  is  generally  a  strict  inequality. 


The  above  summary  corresponds  to  parametric  one-step  prediction;  that  is,  it  corresponds  to 
the  ease  where  die  measure  po  that  generates  the  data  sequences  is  known.  In  this  paper,  we  arc 
concerned  with  the  outlier  model.  Then,  the  observation  process  { Y  ,  -°°<n<°°}  is  generated  by 
direc  mutually  independent  processes;  the  nominal  process  (Xn,  -«><n<°°)  and  two  i.i.d. 
processes  { Vn,  ^<n<°°}  and  (Zn,  -<»<n<°°},  as  follows: 


+  n=  - -1.0.1,. 


;  where  the  common  distribution  of  the  variables  Z  is  unknown,  and  where  (V  ,  -«><n<°°l 

n  1  n  J 

is  a  binary  process.  In  particular,  for  some  given  e  :  0<e<l,  the  latter  process  is  such  that: 


P(Vk  =  0)=  1  — £ 

P(Vk=l)  =  e  (7) 

In  the  outlier  model  in  (6),  {Zn, -»><n<«}  is  called  die  contaminating  process  ,  and 
{ V  ,  -inf<n<°<,}  determines  the  contamination  law.  In  die  presence  of  die  latter  model,  the 
objective  is  prediction  of  the  nominal  datum  xr,  given  the  observation  sequence  y"  ',  for  all  n, 
and  the  problem  formalization  is  then  clearly  nonparametric.  Let  p  denote  the  measure  of  the 
observation  process,  and  let  {gn)2Sn<-,  denote  a  sequence  of  one-step  predictors,  where 

n  -1 

gn  =  g[)(y,  ).  Let  us  then  define. 


(8) 


c>,gn)^nK{ixn-gn(Y1n-,)]2} 

In  (8),  cn(j.i,  gn)  is  the  mean-squared  error  induced  by  the  predictor  gn,  when  die  measure  or  die 
observation  process  { Yn, -^»<n<°o]  is  p,  and  where  Xn  is  generated  by  the  nominal  process 
whose  measure  is  po.  Clearly,  c  (po,  gn)  is  then  as  in  (3),  and  it  represents  the  mean-squared 
performance  of  the  predictor  gn  at  the  nominal  measure  po,  ( that  is,  when  outliers  are  absent). 

Our  objective  is  to  design  a  sequence  (gn}2<n<«  of  predictors  whose  mean-squared 
performance  is  stable  in  the  presence  of  variations  in  the  measure  p  of  the  observation  process 
{ Y  ,  -°°<n<°°).  This  stability  corresponds  to  qualitative  robustness,  and  is  defined  as  follows: 

Given  r|>0,  dicrc  exists  8>0,  such  that: 


# 


C 


np(po,  p)<8  implies  lcn(po,gn)-cn(p,gn)l<ri;  Vh 

In  the  above  definition,  llp  denotes  Prohorov  distance  with  an  appropriate  distortion 
measure  p  on  data  sequences,  and  sequences  [gj  of  operations  that  satisfy  this  stability  arc 
called  qualitatively  robust  at  the  measure  po.  As  found  first  in  [13],  and  later  in  [  1  ],[  14],  and  [16], 
for  the  sequence  [gn]  to  be  qualitatively  robust,  pointwisc  continuity  and  asymptotic  continuity 
in  conjunction  with  boundness,  arc  sufficient.  In  particular,  it  is  sufficient  that:  gn  is  bounded  for 
all  n,  and: 


bC 


\ 

i ( ' 


(A)  Given  finite  n,  given  r[>0,  given  xJf  there  exists  5>0,  such  that. 


y";Yn(xi"'y")_n  '  I  lx -yd  <5  implies  lgn+1(x")-gn+1(y,n)l<r|. 

i=l 


(B)  Given  po  stationary,  given  £>(),  q>(),  there  exist  integers  n^,  m,  some  5>0,  and  for  each 
n>n^  some  A  eR  with  p^(A  )>  1 — r|,  such  that  for  each  xn£An  and  y"  such  that 
inf[a:#[i  :ym(x|'fm  \y"m  ‘)>a]  <  not)  <  5  it  is  implied  that  lgii+,(x")-gn+l(y")l  <  £. 


S;.» 


Given  a  sequence  (gn)  of  predictors  which  is  qualitatively  robust  at  the  nominal  measure 
po,  its  important  quantitative  performance  criteria  are:  (1)  Us  asymptotic  mean-squared 
performance  at  the  nominal  measure,  /imsupcn(po,  gn).  (2)  Its  breakdown  point.  (3)  Its  influence 

n-*oo 

function.  The  breakdown  point  and  the  influence  function  represent  measures  of  resistance  to 
outliers,  and  their  definitions  arc  given  below. 

Consider  the  model  in  (6),  and  let  then  {Z  }  be  a  deterministic  process  with  amplitude  w; 
that  is,  P(Zn  =  w)  =  1.  Let  then  pcw  be  the  measure  of  the  observation  process  {Yn}.  Given  a 
sequence  (gn)  of  predictors,  we  then  define: 

Influence  Function  of  the  sequence  (gn): 

A  c(p£  w,  g)-c4io,  g) 

I  (w)  /  im -  (9) 

8  _ 

€— *0  £ 

;  where, 

A 

c(p.g)  _  /  imsupcn(p,  gn)  (10) 

n— >«> 

provided  the  limit  in  (9)  exists. 

Breakdown  Point  of  the  sequence  { gn } : 

A  i 

e  *  sup{e:supe(pc  w,  g)</imsupF  |Xn}}  (li) 

II — ♦°° 

;  where  cfp,  g)  is  defined  as  in  (10). 

We  note  that  the  breakdown  point  is  the  maximum  frequency  of  independent  outliers  that 
the  prediction  sequence  can  tolerate  asymptotically,  without  becoming  useless,  (that  is.  Ik  (ore  the 
observation  sequences  provide  no  information  about  the  next  process  datum),  where  the 


amplitude  of  the  outliers  is  arbitrary.  Alternatively,  we  can  define  the  breakdown  point  as 


-  A  2 

e  *  sup{e:  /imsup  c(pt w,  g)  < /imsup  (XJ 


If  c(p£w,  g)  is  symmetric  in  w  about  zero  and  is  monotonically  increasing  in  iwl,  then 


e  *  =  e  *.  In  general,  e  *  is  defined  in  terms  of  a  stronger  condition  than  e  *  and  hence 

8  6  6  fc» 


e  *  <  e  * 
s  8 


The  influence  function  represents  the  slope  of  the  function  c(nc  w,  g)  -  c(|io,g)  Fc  (w),  al  the 


e=0  point.  F£  (\v)  corresponds  to  the  asymptotic  mean-squared  error  increase  induced  by  the 


prediction  sequence  (g  }.  when  from  absence  of  outliers  the  environment  shifts  to  e-frequcncy 


and  w-amplilude  ou flier  occurrence. 


The  outlcr  model  in  (61  can  be  generalized  to  i.i.d.  sequences  oi  m-sizc  blocks  of  outliers,  as 


follows: 


;  where  the  sequence  (Vk)  is  as  in  (7),  and  where  the  vector  random  variables  {Z^,^} 
arc  i.i.d.  with  unknown  distribution.  Let  M-Ewm  denote  the  measure  of  the  observation  process 


[  Y  ),  when  the  model  in  (14)  is  present,  and  when  P(Zn  =  w)  =  1.  Then,  given  a  sequence  (gj  of 


predictors,  and  defining  c(|t,g)  as  in  00),  the  breakdown  point,  e  ,  and  the  influence  function. 


I  (w),  that  correspond  to  the  outlier  model  in  (14)  arc  defined  as  follows: 

g,n\  * 


e,  m  _  supie  :  sup c(pMw|n,  g)  <  I iinsupE^  IX~) ) 


(16) 


© 


r 


( 


ic 

i 


Ic 

I 

I 


f 

f 

r 

l 

to 


A 

I  (w)  /  im 

c->o  e 

_ * 

provided  the  limit  in  (16)  exists.  We  also  define  egm  by  replacing  supremuin  over  w  with  /  im 
sup  as  in  the  ease  of  (12). 

III.  OUTLIER  RESISTANT  PREDICTION  OPERATIONS 

We  consider  a  stationary,  zero  mean,  real-valued  process  (Xn,  -«<n<°°},  with  measure  po, 
2  2 

and  E  (X  )  =  a  <  We  also  consider  the  outlier  model  in  (14)  for  the  observation  process 
n 

{Y  -oo<n<o°}.  We  concentrate  on  the  design  of  qualitatively  robust  and  outlier  resistant 
sequences  (g  )  of  one-step  predictors  for  the  process  {Xn,  -°°<n<<»}.  Our  mctliodology  involves 
two  steps:  (1)  A  saddle-point  game  formalization  and  solution  for  the  predictors  g() :  2<n<m+l. 
(2)  A  qualitatively  robust  generalization  of  the  solutions  in  Step  1;  for  the  predictors  gn :  n>m+l. 

In  the  sequel,  we  will  assume  that  both  the  nominal  and  the  contaminating  processes  arc 
absolutely  continous.  We  will  then  denote  by  fo(x")  the  density  function  induced  by  the  nominal 
process  at  the  vector  point  x";  we  will  denote  by  ffy")  the  density  function  induced  by  the 
observation  process  at  the  vector  point  y".  We  note  that  then,  for  n  :  2<n<m+l,  the  class  Fn,  of 
density  functions  induced  by  the  model  in  (14)  is  as  follows: 


Fn=lf:f(yIn',)-(l-e)/o(y1n'')>0:  Vy^'eR""1,  j  f{y\ 


n'1\i  n_1  ii 

)dy,  =  lj 


07) 


-  Construction  of  Prediction  Operations  -  Step  1 

Let  us  consider  the  model  in  (14)  and  one-step  prediction  based  on  observation  sequences 
y"  ',  with  n  :  2<n<m+l.  Given  such  an  n,  we  consider  the  following  saddle  point  game,  where 
F  is  as  in  (17): 

n 


Find  a  pair,  (/  *,  gn*),  of  an  observation  density  function  and  an  one-step  predictor,  such 


7 


that  /  *zFn,  and: 


' VftF  :  c  (/,  g  *)<  c  (/*,  g  *)  <c  (/*,  g  ) ;  Vg 

J  n  n  °n  nv  cn  cn  rn 


(IS) 


In  (18),  the  errors  c  (/\g  )  are  as  in  (8),  where  the  measure,  p,  has  been  substituted  by  the 
corresponding  density  function,  / . 

Consider  a  pair,  (/', g  ').  of  an  observation  density  and  a  prediction  operation,  such  that: 


(/'.  ?„') :  c  n(f\  gn')  =  supinfen(/.  gn) 


/ s„ 


(19) 


,  ,  n-t 

From  the  results  in  [15|  we  then  conclude  that  if  the  operation  g  =  g  (y  )  ,s  poiniwisc 
continuous  and  bounded,  then  (/',  g  ')  s  (f*,  g  *),  and  the  pair  is  a  unique  solution  of  the  game  in 
(18).  We  now  present  a  theorem  whose  proof  is  in  the  Appendix. 

Theorem  1  Let  the  nominal  process  be  zero  mean  Gaussian.  Let  mjy"  ^sB^y"  '  denote 
the  optimal  at  the  Gaussian  process  onc-step  predictor,  when  the  observation  sequence  is  y"  '. 
Let  n  :  2<n<m+l.  Then,  the  pair(/',  gn')  in  (19)  is  as  follows: 

Sn'Cy"  l)=mo(yin  I)min(l,  X.n_,  I m0(y "  V  ’)  (20) 

/'( y,"  *)  =  (1-e)/0(y1n  l)max(l,Xn_11lmo(yIn  ')!)  (21) 

;  where  X.  j  is  unique,  and  such  that: 


.  r  m ,  n-1  n-1 

=1 


Since  the  operation  in  (20)  is  poiniwisc  continuous  and  bounded,  (/',  g  ')  s  (/*,  gn*).  and 
the  pair  is  a  unique  solution  of  the  game  in  (18). 


8 


c 


When  the  nominal  process  is  non  Gaussian,  the  operation  g  '  in  (19)  is  generally  not 
pointwisc  continuous;  thus,  there  is  no  guarantee  then  that  it  will  satisfy  the  game  in  (18),  and  it  is 
generally  qualitatively  nonrobust.  However,  drawing  from  linear  prediction  in  the  absence  of 
outliers,  we  will  adopt  the  operations  in  Theorem  1,  for  non  Gaussian  nominal  procescs  as  well. 
Specifically; 

Let  the  nominal  process  be  zero  mean  stationary.  Let  m  (y"  *)  =  U  '  ,y"  '  denote  the 
optimal  at  the  nominal  process  linear  one-step  predictor  when  the  observation  sequence  is 

n-l 

y.  .  Let  fG  denote  density  of  the  Gaussian  process  whose  power  spectral  density  is  the 
same  as  dial  of  the  nominal  process,  and  whose  mean  is  zero.  Then,  in  the  presence  of 
the  model  in  (14),  and  for  n  :2<n<m+l,  we  adopt  the  following  one  step  prediction 


operation; 


C(-v"  ')  =  mo^  VnhnO,  ^n_,  lmo(y"  ')!  ') 


V, :  \  /G(yrWo,  Clmo(y.n’1)|)=(I-e)’1  (23) 

r""1 

We  note  that  for  e=0,  the  value  of  Xn  ]  is  infinity  and  the  operation  gn*  becomes  identical  to 
the  optimal  of  the  nominal  linear  one-step  predictor.  As  e  increases,  A,  decreases 
monotonically,  becoming  zero  at  e=l. 

Construction  of  Prediction  Operations  -  Sten  2 


In  this  pan,  we  are  concerned  with  die  construction  of  qualitatively  robust  prediction 
operations,  for  large  dimensionalities  of  observation  sequences.  We  point  out  that  die  operations 
in  (23)  are  qualitatively  robust  for  finite  such  dimensionalities  only.  Indeed,  they  satisfy 
condition  (A)  in  section  II  and  arc  bounded,  but  they  do  not  satisfy  condition  (B).  At  the  same 
time,  the  outlier  model  in  (12)  docs  not  allow  for  the  formalization  of  a  saddle  point  game  for 
arbitrary  data  dimensionalities,  even  when  the  nominal  process  is  Gaussian.  We  will  thus  adopt 


9 


an  ad  hoc  approach. 

Let  {a^}  1<J<n  denote  tnc  one-step  prediction  coefficients  of  the  nominal  process,  when  n 

n 

observation  data  arc  available.  That  is,  if  nto(y,)  denotes  the  optimal  at  the  nominal  linear  one- 

n 

step  predictor  when  the  observation  sequence  is  ,  then: 


,  n  (n) 

m  fy.)=  V  a.  y 

o  -  1  7  *-  J 

J=1 


(24) 


Let  gn  be  as  in  (23).  Then,  we  propose  the  following  sequences,  { G j  n*}  and  {G-,n*}  of 
onc-stcp  predictors: 


Sequence  (G1  *) 


Gi/(y,"  ')=GJ(y"  ')  =  £n*(y"  .')  ;  for  2<n<m+l 


*  ,  n-l  *m  n-1  (n-1) 

Gt^y.  )=G..n<yi  )=iaj 

j=i 


SjliCyJ)  -  BjliCyr'- °> 


0) 


n-1 


+  2  »j 

j=rn  +  l 


(n-1) 


gm+1(y-_n  o-CiCi-0) 


(m) 


r  ,  for  n>m+l 


(25) 


I  +k 


where  (y,  ,  0)  denotes  the  sequence  (y; ,  y/+ . y/+k,  0} 


Seouencc  (CL^*] 


,  •  n-1  *m  n-1  *  n-1 

G2 n(y |  )  =  G,„(y1  )  =  gn(y,  ).  (or2<n<m+l 


l,n';l  7  cn  1 


It) 


>(<>.'•) 


*  n-l  _^*m  n-l  (n— 1 ) 

>=  x  \  ■ 


k=1 


Rn+1*(o.y,k.«)-pl,(L».y,k",.a)' 


(m) 

V t(— l.n) 


n-l 

1 - H 


l(i+l,n) 


+  2  I  -k 

i=0  k=l(i,n>+l 


(n-l) 


(m) 

3k-«(i.n) 


,  forn>m+l 


whore. 


(26) 


A  n-l 

t(i,n)  im  +  n-l-(  Jm 
~  m 


(27.a) 


(1}.>V0) 


-[p,0,0,...Q,yry? . y.,p.O,...Oj 

m-l(0,n)  •(U,n>-j 

. 0.0,  (),...,  Q| 

' - - ' 

\  m 


l<j<t(0,n) 


j=0 


(27.b) 


and 


(y  ,Q)  = 


['[yry/+1 . . Qi  /<j</+m-i 

m-(j-/+l) 


[0,0 . 0,0,0 . 0] 

N  ■■  yy.  ■ 


j  </ 


(27.  c) 


If  the  dcnominaior  of  (•}  of  any  term  of  (25)  or  if  the  denominator  of  f  |  of  any  tenn  of  (26)  is 

zero,  then  that  term  is  not  included  in  the  sum. 

*  * 

We  observe  that  the  sequences  { G j  }  of  (25)  and  (G,n)  of  (26)  degenerate  to  the  sequence 
of  the  optimal  at  the  nominal  linear  prcdictors.whcn  in  me  model  in  (14),  e=0,  (design  in  the 

absence  of  outliers  ).  In  addition,  using  a  similar  proof  as  in  1 1  HI,  we  can  easily  show  that  the 

•  * 

sequences  { Oj  }  and  (G,  }  arc  qualitatively  robust,  (satisfying  condition  (B)  in  Section  II),  if: 

tl 


sup  52  1  aj  I  =  c*  <  «  (28) 

k 

J=1 

*  * 

The  sequences  {G1  )  and  {G2n}  arc  identical  for  m=l.  For  m>l,  these  sequences  differ: 

* 

For  n  >  m+1,  the  predictor  Gj  n  is  defined  in  terms  of  the  overlapping  sliding  blocks  of  length  m 

* 

observations,  while  the  predictor  G.,n  is  defined  in  terms  of  disjoint  blocks  of  length  m 
observations. 

Asymptotic  Performance  at  the  Nominal  Process 


In  this  pan,  we  focus  on  the  asymptotic  mean  squared  error  induced  by  the  sequences 

•  * 

{G,n}  and  {G-, n }  at  the  nominal  process  (iQ.  We  will  lirst  assume  that  po  is  a  zero-mean, 
stationary  Gaussian  process  and  evaluate  c(p.o,  G( '")  where 


c(po,  Gt  )  =  c(po, G*)  =  l irnsup e(po,  Gjn  ),  i=l ,2 


Then,  for  a  general  class  of  stationary  processes,  we  will  obtain  an  upper  bound  of  (29),  which 
will  be  light  for  small  e. 

Fix  any  m>  1 .  Given  the  infinite  past,  let  the  nominal,  linear,  optimal  onc-stcp  predictor  be: 


=  I  ty. 


If  is  also  Gaussian,  then  (30)  represents  the  nominal  optimal  onc-stcp  predictor,  given  the 
iQlinitc  past.  Define: 


m  A  •  m  •  m-1 

Pm!Ul  1  gn.uK  -())l 


Vkiu" ’i  t Ok ’ (i» - Q)i- 
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where  gm+!  is  given  by  (20),  uj"  =  (u,  u2,...,u n)eRm  and  (uj,  0)  is  given  by  (27.c). 

Then,  given  the  infinite  past,  the  designed  robust  predictors  in  (25)  and  (26)  respectively 


-i  d. 

„  *rn  —  1  -1  1  *  i  *  i-1 

G,  (y-J  =  c,*(y_j=  s  (m)  ( gm+i f yi-m+, i  -  sm+, f (yi_m+i •  °)i ) 

i=-“ 


-i  dj 

~  2  (m)  P,"tyi-n>+l1 

i=-“  3m 


•m  -1  •  -1 

G2  (y_)  =  G2(y_)  = 


-l  m  d.  .  , 

im+k-1  *  •  i  .  ,  „ 

S  —  (g^IoC  "  .0)-gm+|[(y™  .0)1} 

i=-«  k=i  ak 


-1  m  d.  .  . 

im+r-l  ,1N  * 

_  _  _  .  (i+l)m-l 

'  2  2  (m)  qm'kIy™  1  (34) 

i^o  k=l  ak 

.  (m)  i 

where  ak  arc  given  as  in  (25),  (26)  and  (yj,  0)  is  defined  by  (27.c). 

Let  pp  be  tlie  process  induced  by  po  and  pm  (31).  Then,  p  is  a  zero-mean  stationary 

m 

oo 

process.  Let  (R  (t)}l=_^  be  the  autocorrelations  associated  with  the  process  p  .  That  is, 

'  m  P 


Rr  <H>  -  — <ij<. 


Also  let. 


Similarly,  wc  define. 


)« 


L 


VW 


"t 


m.k 


i-l  (j>l)m- 

1  l<k,/<m 

)q  .  X 

'in,/ '  jin 

)l. 

— «=<i,j<» 

(37) 

...  (i+l)m-l 

i<-l 

(X.  )  , 

.  im 

l<k<m 

(38) 

We  will  express  c(po,  G(  )  in  (29),  in  terms  of  the  quantities  in  (35)-(38).  These  quantities 
arc  non-trivial  to  obtain,  since  the  mappings  pm  and  qm  arc  nonlinear.  Wc  will  determine  these 
quantities  assuming  that  p  is  Gaussian. 


Assume  that  p  is  a  Gaussian  source.  Let, 


Z..  =  m(X.‘  ,)  =  BTX.' 

l.i  i-nH-r  m  1-1 


m+1 


I  a(m>  X. 

"  •  i-m+t 


(39) 


i-i 


in— 1 

Z2.i =  I  at(m)Xi.n 


(40) 


i=i 


Then,  {Z,  t}  and  {Z,.}  are  zero-mean  stationary,  Gaussian  processes.  Let  {R„(j))  and  (R^(j)}  be 
the  autocorrelations  associated  with  these  processes  respectively,  and  let  {pu(j)},  (P-,,(j))  be  the 
associated  correlation  coefficients.  Then, 


Rn(j-i)  -  EtZ.  ZJ  _  pn(j-i)R,,(0)  -«.<i,j<o 


(41) 


A  A 

R22O-')  ^Z2iZ:j^  P22(j-OR_(0)  -oo<i,j<oo 


(42) 


Let, 
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(43a) 


Ri;(j-i)  =  E[Z,  -Z2j]  _  pI2(j-iWRn(0)R22(0).  -~<i,j<° 


R.:,(j-i)  =  ElZjZ^]  _  p2,(j-i)  VRuU))R22(0),  -«<i,j<° 

Also,  let  R  (0)  =  a2  =  E|  X.2|  and 


(43b) 


R(h(i)  =  E[X()Z(  .|  _  poi(i)o  V R|t(0).  t=  1 .2;  i<-l 


(43c) 


Define: 


=  0Sk<m.  is-1 

1=1 

Then,  VV0i  =  0,  and  for  each  k>l,  {Wk;}  is  a  zero-mean,  stationary  Gaussian  process.  Let, 


(44) 


RWw0-i)  =  E[wk.vv/.]. 


k ,/  =0,l,...,m 

— oo<i,j<oo 


(45) 


and  let. 


Rw  o-o 

w 


Pw  0-0  = 

t; 


(Rw  k(0)Rw  (0)) 

k •*  w(.( 


1/2 


^  0, 


if  exists 


otherwise 


(46) 


Let 


R_(i)  =  E[X,Wkt],  0<k<m;  i<-l 


(47) 


15 


p,»  (') = 


R  (i) 


[a-Rw  (0)] 

k.k 


if  exists 


otherwise 


Let  <+>(x)  and  <i>(x)  respectively  be  the  standard  normal  density  zero  mean,  unit  variance,  and 
its  cumulative  distribution  function,  evaluated  at  the  point  x.  Also,  let  0<n\u)  be  the  nth 
derivative  of  w.r.t.  x,  evaluated  at  u.  Fix  any  A>().  Let, 


hx(u)  = 


u,  lul<\ 

Xsgn(u),  otherwise 


For  any  o2  and  p  such  that  c^O,  o,>0  and  Ip  I <1 ,  let  us  define: 


A  2  2-2  ^ 

OfX.o,)  o,<l>(  )  +  (2X  — 


piX.Oj.Oj.pl  A[A.,aI,o,.p]-I|X,o1,l]  + 


+  [o22(l-p2)ll/2  V  F[a1,p,n|-plA.,<32,p,nl  I[X,ornl 


+  A.  £  F[o1,p.nj  p[A„o;,p,n-l)  I[A.,a1,n] 


o  2  1/2 

+  2[o‘(l-p  )1  £  Flo1.p,n]  P|X.a2,p,n-21I|X.a1.n| 


where. 


5T"^ 


$ 

I 

I  .*> 


and  where 


;  x  r  -x' 

A[X,a1,a;.p]=Yt[<I>  -d>  ) 


p[X,a,,p,n)  =  <J> 


F[a,,p,n|  = 


1-p  °i 


I[X,a,,nl  =  J  hx(x)  x"- 


o;  =  o,(l-p*) 


YC  =  P 


Also,  let  £[X,a,,a,,p]  be  given  by  the  r.h.s.  of  (51)  except  that  we  replace  I[X.a,,l]  with 
IKOpll,  and  I[X,a,,n]  with  I[»,a,,n].  Fora,  =a2  and  p=l,  we  define: 


p[X,o1,a,,l]  =  0!X.a1l  (57) 

Notice  that  die  computation  of  0  does  not  involve  any  series  where  as  for  lpl<l,  p  and  C,  are 


given  in  terms  of  series.  The  definition  (52)  is  also  consistent  with  the  meaning  associated  with  0 


and  p,  (see  (B.2)  and  (B.3)  of  the  proof  of  Theorem  2.)  Using  direct  but  tedious  computations, 


we  obtain: 


P[X,o,,p,n]  = 


2]  2  -2  “  ui  X  2k -2, + 1  (2k+l)!2~* 

exp(-X  a.  )£(-!)  (  )  ,  if  n  odd,  n=2k 


t=0 

a 

c 

(2k-2t+l)!l! 

1 

if  n  even 

'1  =  IKo,.n] 

1  =  0.  Ifn= 

=2k+l,  then: 

1 

X  X 

k 

1 

=  - 

(-)“ 

‘j  'few 

—)_<!>(_— 

)J  Il(k-t+— ) 

rK 

a 

l 

a,  o, 

t-4)  2 

2 

-a 

1  2k  + 1 

k 

-21 

1  ] 

— C 

•a  ( 

l+va  ■ 

n(k-t+-)i 

► 

l  =  \ 

2  J 

2k+l 

(58) 


.  I  j+i 

1  k+t  2k+2  -it  a 

+  —•2  a,  k!c  .£ - 


I[«,o1,2k+l]  =  a‘  -  n  (2i-l) 

t=i 


where 


We  are  now  ready  to  obtain  c(p(),Gi  )  in  (29),  assuming  certain  conditions.  Let  N  be  the 
set  ol  nonnegative  integers.  Let  J  be  the  Borel  o- field  generated  by  the  discrete  topology  on  N. 
Let  (NxR,  )XB )  be  the  product  space  where  J XB  is  the  Borel  o-lield  generated  by  the  product 
topology  on  NXR.  Let  v  be  the  product  measure  on  J  XB ,  product  of  the  counting  measure  on  J 
and  Lebesque  measure  on  B .  Deline, 


c 


r(n,\lX,o|,c„p) 


(n)  1  Ye  n  X 

(  )•(  )  #—) 

GO  O , 

c  c  n+1  1 


(62) 


•x 


n! 


where  oc  and  yc  are  given  by  (56. a)  and  (56. b)  respectively. 

Theorem  2:  Let  be  a  zero-mean,  stationary  Gaussian  source  with  variance  a".  Assume 
that  (28)  holds.  Then, 


-i  d.  -l  -l  dd. 

•m  •  2  1  1  J 

CKG1  )  =  c(n0.G,)  =  o  -2  1  -77Rxp(i)+  z  X  — 7RpG-i)  (63) 


(m)  *P~ 

'=-~  3m 


(m)  2  pm 

j—  K  I 


-l  m  dm+k-1 


e(po,G2  )  =  c(po,G2*)  =  o-2  £  X 


- R  0) 

(in)  4m.k 


i»-“>k-l  ak 


-i  -imm  dm+k-1  d  ,  , 

i  jm+/-l 

- • - R  (j-i) 

(m)  (m) 

a, 


+  I  I  II 


(64) 


i=-°»  j=-°°  k=l  /=!  “|t 


(m) 


where  (d  },  a  are  given  as  in  (33)  and  (34),  and  R  ,  R  ,  R  and  R  are  given  by 

1  *  D  xn  a  .  .  xa  °  J 

(35)-(38).  If 


’I’m  Vw  *<W 


|r(n,x  I  X,arcrrp)dv 


(65) 


> 


K 

\ 

» 

I 


exists  (i.c.  the  integral  is  not  of  the  form  <»,  jn  the  sense  of  Lebesque)  for  all  tuples  (\,aro2,p) 
which  are  the  arguments  of  (i  and  ^  of  (66)  and  (67)  below,  then,. 

2 

Rpn0-i)=  I  (-1)  PI*,,.  \R„(«).  VR  ,(<>).  pa(H)!.  -00<>-j<°°  (66) 

/,i=i 


and  since. 


(n) 

sup  f  4>  (x)l  < 


<H  ) 


•( - )! 


n>l,  n  odd 


dx  =  <5, 


n>1,nodd 


hence. 


2  •( - )! 


>  •  P 

£  j  lr(n,xl?.,o1,o2,p)ldx£  £  (  )  (n+1),  n  odd 

n=l  n=l  ^ 


The  scries  in  the  upper  bound  of  (73)  converges  if. 


Similarly,  summing  over  even  values  of  n  of  the  series  in  (70),  we  arrive  at  (74).  Hence,  (74)  is  a 

*m 

sufficient  condition  for  the  existence  of  the  integral  in  (65).  In  this  case,  cQi^.Gj  )  can  be 
approximated  by  considering  only  a  finite  number  of  terms  of  all  the  scries  of  p  and  £. 


(2)  Holder  inequality  gives  a  simple  condition  for  checking  whether  the  scries  in  the  definition 
of  P  or  £  is  divergent.  Define  N  ,  N,  as  in  Lemma  I  of  the  Appendix.  Then  by  (B.3)  and 
Holder, 
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iP(X,a1,o2,p|l  =  I  E[hx(NI)hx(N,>]  I 


<  (E[hx(Np]YPNE[hx(N2)]Y/2 


=  {O[X,OJ},/2{O[X,0J),/2 


The  upper  bound  of  (75)  does  not  involve  any  scries  and  can  be  easily  computed  using  (50). 
If  the  series  of  (i  diverges,  then  (75)  will  not  hold.  Similarly, 


lQX,o1,o,.p|l  < Oj  fOlA.a^l) 


We  will  now  obtain  an  upper  bound  if  c(jioG.  ),  which  is  also  applicable  to  non-Gaussian 

processes,  which  does  not  require  any  restriction  as  that  in  (65),  which  is  easy  to  compute,  and 

♦  * 

which  is  directly  related  to  e(po,mo),  where  c((io,mo)  is  the  asymptotic  mean  squared  error  at 
that  is  induced  by  the  nominal  optimal  linear  predictors  mo(y"  ')■  Using  the  proof  of  Theorem  3 
below  an  upper  bound  of  e(p  ,G.,m)  can  also  be  obtained  in  a  similar  manner.  Let, 


H  =  E[(l  X  1  + 

in  v  m 


- )“{ 1-1  (X  )}1 

(m),  J 


where 


Also,  let 


m  in  (m)  „  (m) 

J  =  {y  :  I  V a  y  I  <\  and  I  T  a.  y.l  <\  ) 

J  j  J  j  m  j  '  1  m J 


D*  =  £  Idd 


Theorem  3:  Let  { X  }  be  a  zero-mean,  finite  variance  cf,  stationary  source  with  distribution 


u  .  Fix  anv  m>l .  Then. 

rO 


U>>  e(Ho.G1,")  =  c(H0.G| )  <  (!c((.i).m  ))|1/2  +  D*yjll  f 


l  im(  |c(no.ino)i'/2  +  D*^I T)‘  = 


Proo I':  See  the  Appendix. 


Remark:  For  Gaussian  sources,  if  m=  1 ,  then. 


u 

“  <H  ) 

2\  ,  a 

11=2  [  (u+  )”•  du 

0)  „ 

x,  'a,  I  a 


For  ni>l,  we  obtain  an  upper  bound  of  Hm  which  is  easy  to  compute,  as  follows.  By  (C.7)  in  the 
Apppendix,  we  have: 


E[X^(1-1  jm(Xm))]  <  {E[XJ ) ,/2- {E[ l-lr(Xm)l } 1/2  = 
=  V3a2  {E[l-ljIT,(Xm)]}1/2,  t=l,2 

t 

E[IXml(l-ljn,(Xm)]<a(E[l-ljm(Xm)l}1/2,  t=l,2 


Therefore,  by  (77)  and  (C.6)  in  the  Appendix,  we  obtain: 


H  <^2<  '3a  + 


4  A.  a  4X‘ 


i  (m)i  i 

la  I  la  I 


!-<!>( - +<  l-<b 


Ill- l+l 

2  (m)  _ 

where  a,  is  the  variance  of  V  a  X,  j=l,2. 

t.m  —  j  j  J 

j=i 

IV.  BREAKDOWN  POINT  AND  INFLUENCE  FUNCTION 

In  this  section,  we  obtain  tite  breakdown  points  and  the  influence  functions  of  the  prediction 

operations  in  (29)  (m=l  only)  and  the  operations  in  (30),  following  the  definitions  (9)-(l 6).  We 

first  consider  the  case  of  the  per-datum  outlier  model  in  (6)  and  the  operations  in  (29)  and  (30), 

for  m=l.  Then,  we  consider  the  case  of  m>l.  It  is  easy  to  verify  that  the  breakdown  point  of  the 

2 

nominal  optimal  linear  predictors  in  (24)  is  zero,  for  any  zero-mean,  finite  variance  a  ,  stationary 
process,  po.  Let  I01(w)  be  the  influence  function  of  these  nominal  predictors  for  the 
contamination  model  in  (6).  Then, 

-i  -l  -l  -t 

l„.,(w)  =  (wV)- 1  d ‘  -  2  I  v  d,d  R/j-i)  +  2  z  d,Rx(i)  (84) 

J=— OO  i=— oo  j=— CO  1=3—00 

—  1  oo 

where  is  given  by  (30)  and  {R^i)).^^  is  the  autocorrelation  function  of  the  process  po. 

Consider  now  the  predictors  in  (29)  and  (30),  which  arc  designed  assuming  that  the  nominal 
process  is  |A  and  the  level  of  contamination  is  e.  Asymptotically,  these  predictors  arc  given  by 
(33)  and  (34).  Let  ni=l.  Then  (33)  and  (34)  coincide.  Let  G*  denote  the  asymptotic  predictor  in 
(33)  (or  (34))  for  the  case  of  nt=l.  Fix  any  w  and  let  the  contaminating  process  be  deterministic 
with  amplitude  w.  Let  the  level  of  contamination  be  5.  Then, 
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I  vi«."m«"  ***  m'’ 


,*S 


k 


k 


Jv1 

£ 

s 


is 


£ 


WV 


B 


«*- 

tV 


E 


Pic 


E 


-i  d 


=  n[{X0-G*(Y;j}2l5.w|  =  E[{X0-  £  - -p,(Yi)}2l8.wl 


(i) 


-i  d. 

=  o:-2S  — •E[X0pI(Yi)l5.w]+  X  £ 


i  -i  d  d 
>  j 


(i) 


(i)  (i) 

3  3 

i=~oo  j=^oo  ul  UJ 


■E[pI(Yi)p1(Yj)l8.w]  (85) 


Now,  for  any  i<-l  and  by  (6),  we  have, 


E|X0-Pl(Y.)  1 8,w]  =  (l-S)E[X0p1(X|)]  +  8E[X0-Pl(w)J 


=  (1-5)%  0) 


where  R  (i)  is  defined  bv  (36).  Also,  for  i=j<-l,  wc  have, 

*p,  * 


E[p|(Y.)Pl(Y.)  1 8,w]  =  E[ (p,(Y.)}  1 8,w)  =  ( 1 -S)Ef p,( X()]“  +  8E[p,(w)f 


=  d-5)Rp  (0)  +  8[p,(w)|* 


and  for  i*j 


E[p|(Yi)p1(Yj)l5,w]  =  (1-5):E[p1(Xi)p1(Xj)]  +  52|p1(w)]2 


+  5(l-5){ElpI(Xi)p1(w))  +  E|p1(Xj)p,(w)J) 


=  (1-5)  R  0-0  +  5  (P,(w)l 


where  {Rp  (j— i ) }  is  defined  by  (35). 


(86) 


(87) 


(88) 


Using  (86)-(X7),  we  can  determine  the  breakdown  point  in  (1 1)  of  the  predictors  in  (29)  (or 
(30)),  for  m=l,  as: 


e.->  =  eG.=  sup  (5:  supe<pfiw,G,  )<c~ 

1  (>Sd<l  w 


(89) 


Equivalently, 


25 


AM 


e 


G* 


(1-S)R  (0)  +  5A., 


-i  -i  d  d.  -l  d. 

+  2  -  +  -^%(0)  (90 

i=— “j=-~  ai  i=-«.  ai 

•  _• 

Notice  that  for  m=  1 ,  eG,  =  eG,  (( 1 2)). 

VVc  now  detennine  die  inlluencc  function  substituting  (S6)-(8S)  in  (X5).  wc  obtain: 


C(fl6,w*c+)  ~  C(P„>G*)  = 


=  E[ ( X()-G *( Y^) )' 1 1 8,w |  -  E[ X()-G*(X~' )]2  = 


-i  -l 

+  I  I 


-l 

=  25-1 


-l  d, 

L  -^••{E|X0p1(Y.)l8,w|-E|X0-p1(X.)|} 


d,  d 

— • — •  ( E(p,(Y  )p  (Yj)  1 6,w]  -  Efp,(X  )p,(X.)J } 

0)  (i)  1  '  1  J  1  1  J 


d.  -i  df 

—  R  „  (0  +  52  ( - )Z-{lp,(w)]2  -  R  (0)} 

0)  *pi  (i)  1  pi 


a 


i 


-i  d,  d 

+  2  E  ~  '  -l^iPifw))2  +(-25  +  52)R  (j— i)}  (91 

(i)  0)  pi  v 

i=^»  j=-«.  a|  a| 


Therefore,  the  inlluencc  function  I  (w)  lor  the  designed  predictors  in  (29)  (or  (30)),  form=l,  is: 


-hi  T«  mil  ‘hi* - 


(92) 


-i  d. 

I  .(w)  =  2  £  “ 
(l  (i) 


-l  -i 

-2  1  I 

i=-oo  ys—oo 


d.  d. 

—  ~  R  (j-i) 
(i)  0)  p> 

31  3i 


Notice  that  fore>(),  I  .,(\v)  is  a  bounded  and  continuous  function  of  w,  in  contrast  to  I0)(w), 
in  (S4).  Also,  if  £=(),  then  (92)  reduces  to  (84). 

The  above  expressions,  (90)  and  (92),  of  the  breakdown  point  and  the  inllucncc  function 

oo  —  ] 

respectively;  require  the  knowledge  of  {Rp  (i)}^  aa  and  (Rxp  (i))i=  <>j.  If  is  also  assumed  to  be 

Gaussian,  then  we  can  determine  those  quantities  using  Theorem  2,  if  (65)  holds.  For  non- 

Caussian  sources,  or  for  Gaussian  sources  where  (65)  docs  not  hold,  it  may  not  be  possible  to 

determine  (R  (i))  and  (R  (i)}  analytically, 
p,  xp. 


We  now  take  an  alternate  approach  and  determine  an  upper  bound  of  the  influence  function 
and  a  lower  bound  of  the  breakdown  point,  for  nominal  processes  that  arc  not  necessarily 
Gaussian.  We  have 


Now, 


-t  dj 

c(p.w.G*)  =  E[{X0-G*(Y^)}2l5,w]  =  E[X0-  £  “PA)]2 

i=— “>  3l 

-i  d(  -i  d. 

+  2E{[X  -  V  —  p.(X.)]-[  S  — (Pl(X)-p  (Y.))]l5,w} 
w  (t)  1  1  (t) 

ai  l=— «■>  ai 

-1  d( 

+  E[(  v  -L(Pl(X)-pI(Y))}2l8,w| 

(i)  ' 


(93) 


t 


C 
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c 


lc 


ic 

5 


-i  d(  -i  ci, 

E{[X0-  I  -p1(Xi)HI  —  CP.CXjJ-p^Yj))! 1 5.w } 

i=-~  U1  i=— •  ai 

-i  d.  -l  d. 

=  5E{[X  -  ^  (X)]  (  X  —  (p,(X  )-Pl(w))l} 

0  <D  0)  1  '  1 

i=-oo  ai  i=-oo  ai 


-i  d.  -1  d. 

=  5E{[X0-2  — p,(X.)]-[  £  —  Pl(X-)]) 
0  (i)  (i) 

i=-oo  ai  i=-oo  a] 


-i  d.  -i  d. 

<5(E[X0-2  -^Pl(X.)n1/2-{E[  £  ~p1(Xi)|Y/: 


i=—  a, 


i=-«o  ai 


-i  d.  -i  d 

1  ,2  1/2,  1 


ss  (E(x0-  z  —  p,<x,)]  )  l  s  i  —  i R, (0)i 

a. 


l=~aa  aj 


(1)  ‘  “  (1)  P> 


(94) 


The  last  two  inequalities  of  (94)  follow  respectively  by  Holder  and  Minkowaski.  Also,  by 
Minkowaski  we  have: 


-t  d. 

(E[{  £  “"(PjfXj) -  Pj(Y-))}2 1 5,w) } 1/2 
(i) 

i=-~  ai 


-t  d. 

1  2  1/2 

<£  I - l-{Hf{p1(Xi)- Pl(Y.)}  15, w]) 

(l)  1 

ai 


-1  dj 

=  8,a  £  |-J-|  (E|p  (Xi)-p  (w)|2)'/2 
(i)  11 


(95) 


Therefore, 
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(96) 
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3 

n 


a 


13 


\i 


3 


H 

'* 

S 

•t 


a 


SEES 


-i  d. 


E[{  X  - (P1(Xi)-p,(Yi))}‘l5.w]<S5(2:  I - I)  (Rp(0)  +  [Pl(w)f) 


t=— *»  al 


i=-oo 


(l)  ‘l 


Using  (93),  (94),  (96),  and  Theorem  3,  a  lower  bound  eG„  of  the  breakdown  point  eG.  is 


obtained  as: 


-i  d 


eG.  =  sup  (5  :  (w'c(po,mo*)  +  D*a/H,)2  +  2  S  (  £  I - I  -Rp  (0)KVc(HoJno*)  +  D^H,). 


i=-oo 


0<5<1 


-i  d. 


+  5-(X  I - 1)  (R  (0)+X.“)<o  } 

(i)  p>  1 


,=-  ai 


For  small  c,  the  lower  bound  e  is  strictly  positive  by  Theorem  3b.  Also,  an  upper  bound  I  [w] 


of  die  influence  function  I(,,[w]  in  (92)  is  obtained,  using  (93),  (94),  (96),  and  (80).  The  bound  is 


as  follows: 


Io.[w|a2(>/c(p0jn0*)  +  D*>/H^)-[X  l“IRPl(°)l+  S  ■(Rp,(0)  +  lpi(w)r) 


-t  d 


The  upper  bound  Ic.[w|  is  a  bounded  function  of  w,  if  e>0. 


We  now  consider  the  case  of  m  >  1,  and  for  die  predictors  in  (30).  we  determine  the 


breakdown  point  and  the  influence  function  assuming  that  the  observation  process  corresponds  to 


the  m-si/c  (block)  outlier  model  of  (14).  Fix  any  w  and  let  the  contaminating  process  be 


deterministic  with  amplitude  w.  Let  the  level  of  contamination  be  5.  Then,  by  (34)  and  the 


model  in  (14),  and  following  the  steps  (86)-(88),  we  obtain: 


C^8.w,m-G2  )  = 


-1  m  d 


:E[{X  -G,’  (Y^)}  l5,wm)  =  o*-2(l-5)  £  £ - —  R  (i) 

“  (m)  *  iiiji 


i=—  k=l  \ 


-1  m  m  d  .  .  d  ,  . 

nn+k-1  im+/-l 


m .  .  m . 


+  1  I  I  '  {(1-5)R  (0)  +  5(q  |w  lq  [w  |)) 

(m)  (in)  Vk.i  ,nK  '' 


i=-o»  k=l  /  =  !  \  az 


-1  -1  m  m  d  .  d  ,  , 

iin+fc— 1  jin+/-I  ->  ->  m  m 

+  I  I  Z  0-i)+5‘(qinklw  J-q  [w  |)}-  (99) 

(m)  (m)  Vk ./ 


i=-> »  j=-oo  k=I  /  =  !  l,|c  a/ 


In  (99),  wm  is  (lie  iengih-m  vector  with  all  its  elements  being  w.  Also,  we  have  used  wm  in  the 


conditioning,  to  indicate  that  the  contamination  model  in  (14)  is  used.  Using  (99),  we  obtain  the 


breakdown  point  e  -m  ,  (15),  and  the  influence  function  I  .m  [w|,  (16),  directly  by  the  definitions 


below. 


£G*mm=  SUP{5:  SUpC(PS.w.nA 
2  '  0S8<1  w 


-1  m  d. 


I  [w]  =  2S  Z— Rxq  0) 


(m) 


i=— °*»  k=l  “k 


-1  m  m  d  ,  ,  d  ..  . 


^7,“IT((q»‘|w"lq'"i|w"l"R»  „(0» 

(m)  (m)  ^mXJ 


,  =  ^ok=l/  =  l  \  a/ 


-1  -1  m  m  d  ,  .  d  ,  , 

im+k-1  jm+/-l 

-2  1  z  z  i  — ■— 77  Rq  <i-i> 

(in)  (m)  Mmk.i 


J=_»  k=l  l  =  \  Uk  a/ 


For  non-CIaussian  sources,  an  upper  bound  of  the  inllucncc  function  and  a  lower  bound  of 


the  breakdown  point  for  m  >  1  and  for  the  block  contamination  model  (14)  can  be  obtained,  using 


tf 


» 


l> 


p 


I 


t* 

I 

I 

L, 


Theorem  3  and  steps  similar  10  those  taken  to  obtain  the  results  forin=l. 

For  m  >  1,  for  the  pcr-dalum  outlier  model  in  (6),  and  lor  the  predictors  in  (29)  which  arc 
designed  assuming  the  level  of  contaimination  is  e,  e>(),  we  have: 


-i  .2 


c(H5wl.Gt  )  =  E[{X0-Gl*(Y_J}‘l5,wJ  = 


-i  d 


-i  -l  dd 
'  j 


•o-i  S  -r:|0X  p  <Y'  j 1  s.wi  +  j  s  — •cip,„<Y,'n„l)p,>(Y^,1)i8.w| 

(in)  ,  (m),J 


a 

i^00  m 


; _ _  3 

i=-°°  J=-“  '  m 


(102) 


Hence, 


supc(n6  w ..  Ci,  )  (103) 

w 

is  continuous  in  5  at  5  =  0.  Also,  if  5=0,  then  c(j.igw  ,,G,  )  equals  the  asymptotic  mean  squared 
error  cfp^.Gj*)  induced  by  the  predictors  {Gln}  at  llic  nominal  process  po.  Hence,  by  the 

definition  of  the  breakdown  point  in  (1 1),  the  breakdown  point  of  the  designed  predictors  {G^J 

* 

that  corresponds  to  the  pcr-dalum  outlier  model  in  (6),  viz.  e  -m  in  (11),  will  be  positive  if  and 

Gi 

only  if 


caio,G]m)<o‘  (104) 

Similar  conclusions  can  be  drawn  for  any  size  (batch)  of  outliers  using  the  model  in  (14). 

V.  NUMERICAL  EXAMPLES 


p 


Let  )t  be  a  zero-mean,  stationary,  Gaussian  process.  Also,  let  be  auto-regressive. 
Consider  the  following  representations  oft  he  nominal  process  )t  : 
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Nominal  Process  2: 


x  =0.7x  -0.3x,  _  +  w,  k=. ..,-1,0,1,... 

k  k-1  k-2  k 

Nominal  Process  3: 

xk  =0.7xk  )  -0.3xk  2  -t-0.2xk  3  +  wk  k=...,— 1 ,0, 1 .... 

Nominal  Process  4: 

xk  =  0.7xk  l  -  0.3xk  2  +  0.2xk  3  +  0.1  xk_^  +  0.05xkS+wk  k=. ..,-1,0,1,... 

In  all  the  four  processes,  { W  }  is  a  zero-mean,  unit  variance,  i.i.d.  Gaussian  process,  such  that  \V 

^  K 

is  independent  of  {Xj)^_V  We  summarize  the  results  for  these  processes,  corresponding  to  tlie 
designed  predictors  in  (29)  and  (30),  for  different  values  of  m  and  for  different  values  of  e,  in  the 
following  tables  and  figures. 

Tables  1,  2  and  Fig.  1,  Tables  3-5  and  Figs.  2,3;  Tables  6-8  and  Figs.  4-6;  Tables  9-1 1  and 
Figs.  7-11  correspond  respectively  to  the  nominal  processes  1,2,3  and  4.  Tables  1,3,6  and  9  give 
the  asymptotic  mean  squared  error  (amsc)  at  the  nominal  process  1,2,3  and  4  respectively,  for  the 
designed  predictors  in  (29),  and  for  different  values  of  e  and  m.  Tables  1.4,7  and  10  give  the 
amsc  at  the  nominal  process  1,2,3  and  4  respectively,  for  the  predictors  in  (30),  and  for  different 
values  of  e  and  m.  Tables  2,5,8  and  1 1  give  the  breakdown  points  c*  and  e*  of  the  predictors  in 
(30),  (and  the  predictor  in  (29)  for  nt=l ),  for  different  values  of  e  and  m.  Figures  l  to  11  give  the 
plots  of  the  influence  functions  corresponding  to  the  predictors  in  (30),  for  different  values  of  e 
and  m  and  for  all  the  four  nominal  processes. 

From  the  above  tables  and  figures,  we  make  the  following  observations: 


(1)  When  m=! .  tlicn  lor  any  e,  the  anise  at  any  ol  the  nominal  processes  1,2,3,  or  4  is  the  same 
for  the  predictors  in  (29)  and  (30).  as  expected. 

(2)  If  the  nominal  process  is  a  pill  order,  auto-regressive,  zero-mean,  stationary,  Gaussian,  then 

for  both  the  predictors  in  (29)  and  (30)  and  for  all  e  values  and  all  m>p,  the  amsc’s  at  the 

•  • 

nominal  process  coincide.  Also,  the  breakdown  points  e  ,  e  and  the  inllucncc 

*  (i2  ,m  (»2  ,m 

functions  coincide  as  well. 

*m 

(3)  For  any  m>l,  the  am  sc  at  the  nominal  process  |.l  viz.  c(^o,G  ),  i=l,2  converges  to 

*tn  2 

c(p  ,mo*)  as  e— ►().  As  e-»1,  c(p o,G|  )  converges  to  a  .  Except  for  the  nominal  process  1, 

•m  2  2 

c(p  ,Gi  )  first  increases  with  e,  exceeds  o  and  then  decreases,  converging  to  a  as  c— >1. 

*rn  2 

For  the  nominal  process  1,  c(p  .Gj  )  first  increases  with  e,  exceeds  a  ,  then  decreases  with 

2  2 
e,  becomes  less  than  o  and  again  it  increases,  converging  to  a  as  e-»l . 

*m 

(4)  For  most  values  of  e  and  m,  the  predictors  (G,  n)  in  (30)  have  smaller  amsc’s  at  the 

*m  *m 

nominal  process,  tlian  the  predictors  (Gj  n ).  Also,  for  the  predictors  (G.,  n } ,  Ific  amsc  at  the 
nominal  process  decreases  with  m;  however,  not  necessarily  in  a  monotone  manner. 

*  —  ..‘m2 

(5)  Thebreakdownpoim.se  and  e  are  positive  il  and  only  il  ctji  ,G,  )<a. 

G  .in  Gj  .in  °  * 

(6)  The  breakdown  points  e  .ra  coincide  for  all  values  of  e  and  m  and  for  all  four  nominal 

Gj  ,m 

processes  except  in  the  case  of  the  nominal  process  3,  with  m=2  and  the  nominal  process  4 

with  m=4.  In  those  cases,  and  as  observed  by  figures  5  and  10,  the  influence  of  the  outliers 

_ • 

is  not  maximum  when  w=°°.  Also, e  *<E  ^  . 

,m  Gj  ,m 

(7)  Typically,  lor  any  e,  the  breakdown  point  e  .ra  is  larger  for  large  m  than  for  small  m. 

G2  ,m 

(X)  For  anv  m>l,  starting  from  zero,  the  breakdown  point  e  .m  first  increases  with  e,  and  then 

Gj  ,m 

it  decreases  with  e  to  zero.  A  plausible  explanation  is  as  follows:  The  breakdown  point  of 
the  designed  predictors  (G.,™)  is  determined  mainly  by  two  factors:  o'-  e(po,G,m),  and  A,  . 


For  large  breakdown  point,  the  threshold  Xjn  should  be  small  so  that  the  inlluencc  of  the 

2  *m 

outliers  is  small,  and  o  -et^i  .G,  )  should  be  large  so  that  even  in  the  presence  of  any 

*  **  • 

level  of  contamination  5  such  that  5 <c  ,  the  quantity  cf-supem,  G  remains 

<t2  .in  1  r  o.w.m  2  ' 

w 

2  *rn 

positive.  In  our  case,  when  e  is  smalt,  then  both  c  -c(p  ,G,  )  and  the  threshold  X  are 

°  -  rn 

2  *,n 

large.  When  t  is  large,  then  both  X  and  a  -c(j.iu,G,  )  arc  small. 

(9)  For  any  m>l  and  w>(),  the  inlluencc  function  1  |w|  is  a  decreasing  function  of  e.  Also, 

(i  ,m 

for  any  ni>l  and  £>0,  the  inlluencc  function  is  a  bounded  and  continuous  function  of  w. 
Fore=0,  the  influence  function  is  not  bounded. 

(10)  For  the  designed  predictors  in  (30),  the  influence  functions  1  .m  fwj  arc  not  always 

G  ,m 

monotically  increasing  for  positive  w  (sec  Figures  5,8,9  and  10).  This  is  because  die 

,  n> 

predictors  G, ni\v  ),  when  treated  as  functions  ol  w  only,  are  not  monotonically  increasing 


VI.  APPENDIX 


Proof  of  Theorem  1:  Take  any  contaminating  density  hn_ ((y "  ').  Let  ffx^.y"  ')  be  die 


density  of  fX^.x"  ').  Let  f(xn,y,  )  be  the  joint  density  of  X  and  y"  .  Then. 


f(x  ,y"  >  =  (1— e)l  ( x  ,y"  ')  +  ef  (x  )li  (y"  ') 

v  n  J  i  '  v  o'  n  J  1  '  o  n7  n-l  J  1 


f(y"  1)  =  u-€)ro(y"  ')  +  E:hn-i(y,n ') 


n-1  n-1 

Now,  for  anv  n  (v.  )  and  anv  n  .(y.  ),  we  have 

-  *-  n*  1  -  n-1-'  1  7 


El  (X„-gll(Yp,)|i  =  JJl«„-g„(yr''f((x„,y"'lW«n>ly"’' 


■jiK^rvKx.'yr’xMnyr'wy 


n-1  n-1 


Rv  (A. 3),  for  any  given  ftxn,y"  '),  the  optimum  gn*(y"  ')  is:  gn*(y"  *)  =  E|Xnly"  ',| 


n  _  j  n— 1 

where  the  conditional  density  f(x  ly.  )  is  used  in  evaluating  E|X  ly  ].  Also, 


maximizing  (A.3)  over  hn((y"  is  equivalent  to  maximizing  it  over  f(y"  ')  of  (A. 2). 


Hence,  using  g^fv"  )  in  (A.3),  by  (A.l)  and  since  EfXJ  =0,  we  have  for  any  bn  l(y"  ') 


inf  E[{X-gn(yin_,)}2| 


*,(*i  > 


n-1  .  n-1 


=  M  V"F~'X„'y"  Dhxjy"  )dxn]f(y"  )dy 


=  0“  -  jlKf(xnly1n"l)dxnrf(yp,)dy1n""  = 


n-\  .2 


=  a2-j- 


[fxfix.y.  )dx 

1 J  n  n;l  7  n 


f(yr') 


=  o2-\- 


I  (  1  _£^0(y l"  '>JXnfo(Xn  1  y l"  ')t,X„  I 


n-1 

f(y,  ) 


n-l  n-1  ,2 

[d-p)f  (y  )m  (y  )] 

2  r  n-1 

-J  r;  dy' 

f(y,  ) 


Therefore,  our  objective  now  is 


|  ( 1  -e)  fo(y  *  )mo(y  * )  I*" 

n-1  p  .  n“* 

f*  =  f*(y(  ):  inf  J - dy, 

fty," ')  fey,""') 


where  fly,"  )  is  of  the  form  (A. 2),  with  constraints 


a.  f(y"-1)  ~  (l-e)f0(y"_1)  -  0  ^  y"'' 


b-  Jf(y,n-1)<Jy"_1  =  i 


We  will  use  the  Lagrange  multiplier  technique  of  the  calculus  of  variations  to  determine  f*. 
The  Lauranue  functional  without  the  constraint  (A. 6)  is 


n-1  n-1  2 

[O-ejf/yi  )mo(y  )] 

J[f*,5|  =  | - dy""  - cxnJ[ )  +  8p(y"  )ldy, 

f*fy"  ')  +  5p(y"  ’) 


where  a  is  the  Lagrange  multiplier.  Hence 


36 


DJ_  I  =() 
cl5  '  6  =  0 


n-1  n-1 


r  n-i  i: 

J<Jy,  p(>'j 
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By  a  fundamental  theorem  of  tlic  variational  calculus. 
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By  (A.  10)  and  the  constraint  (A. 6),  we  have 
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Hence,  by  (A.l  1)  and  (  A. 12) 
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Proof  of  Theorem  2:  The  proof  is  based  oil  the  following  lemma. 

1  .cm m a  1:  Let  N  and  N,  be  two  jointly  normal  random  variables  with  mean  zero  and  variances 
o~,  c’  respectively.  Let  E|N.N,]  =  pr,o,.  For  any  ?■>(),  let 


h.(u)  =  uminj  1, 


l 


I  u  I 


(13.1) 


Assume  the  integral  of  (65)  exists  for  the  tuple  ().,a.,a,.p).  Then, 


E|\(N|)f  =  0[X.a1 


Elhx(N,)lix(N2)]  =  p|A.,o1,a,,p] 


1  2’1 


E|N'IIn(N2)|  =  Cl^1.o2,pl 


(B.2) 

(B.3) 

(B.4) 


where  0,p,  and  C,  are  defined  in  Section  III. 


Proof  of  the  Lemma:  (B.2)  follows  by  direct  computations.  Define  a  ,  y  as  in  (56).  Then, 


x 

°t 


L|hj(N,  )li.  (N,))  =  J  iu(x)  E|lix(N;)IN1  =  xj-  dx 


(B.5) 


ft, 


Also, 
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Substituting  (B.7),  (B.8)  in  (B.fi)  and  in  turn  (Ii.fi)  in  (B.5),  we  get  the  desired  results  (B.3) 
using  the  Fubini's  theorem,  which  is  applicable  if  the  integral  of  (fi5)  exists.  Similarly,  assuming 
this  integral  exists,  we  get  (B.4)  using  (B.fiMB.K). 

*tn  *  in 

We  will  now  g-vc  the  prool  ol  Theorem  2  lor  the  mapping  G  .  The  prool  lorG,  follows 
in  a  similar  manner  Hie  asymptotic  mean  squared  error  etp  .Cij  '")  at  the  nominal  source  p  is 
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Xp  v  1  0  1  m  i-nn-l/J 


=  E|X0h.  (Zli)i-E[yohx(z  )|.  i<-l 


(B.13) 


The  desired  results,  (61)-(b7),  follow  directly  from  (B.l  1 ) - ( B .  1 3 ) ,  Lemma  1,  and  (57). 


Proof  of  Theorem  3: 


(a)  By  the  Minkowaski  inequality  and  the  definition  of  in  *  in  (30),  we  have: 


(e(u,G1'm)},/2={E(X0-G1'm(xilV/2 


*  -1  *  -t  *m  -1  2  1/2 

=  {E|(X0-mo(X_))  +  (mo(X_)-G1  (X_J)|  ) 


-1  2  1/2  -1  *m  -1  2  1/2 

<  {ElX()-mp*(X_J]  }  +  (E[monX_J-G1  (X_J]  )  = 


=  Vc(fto-1V)+i  e  I^(x, 


P,n<Xi-,n+,) 


By  die  definition  of  p  (31)  and  J  (78) 
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Therefore,  by  (('.  I ),  3)  and  the  Minkowaski  inequality,  we  obtain. 
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By  the  stationarity  ot  the  process  P0.W  of  (C.4)  equals  Hm.  Hence,  by  (C.  1),  (C.4)  and 


(79),  we  conclude: 


e(u  ,G  '")  i  |(e(u  ,m  ))  ~  +  D*yJ\l 


(b)  By  (a),  it  suffices  to  prove  that 
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By  the  definition  of  J  in  (78),  wc  have, 
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where 
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j=i 


(C.7a) 


(C.7b) 


m  m 

Now,  as  e— >0,  A.  — ><»  and  hence,  1-1  m(x,  )  decreases  to  zero  as  e— >0  lor  all  x,  .  Since  X 

m  j  1  I  in 

has  finite  variance,  hence  by  the  dominated  convergence  theorem  we  have: 


/ imE|X^(l-l  „,(X1m))i  =  0 
s-*o  J’ 

in 

Also,  by  the  definition  ot  J,  in  (C.7a),  we  conclude. 


j=i 


Therefore,  as  in  (C.8),  we  lutvc. 


(C.8) 


(C.9) 


/  im 
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Similarily, 
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- •  v  I  a.'  I  E(  I X  X.  1(1-1  ...(X,  ))!  =  () 
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(C.ll) 


Hence,  by  (CSV  (CIO),  and  (C.  1 1),  ihe  firs!  term  on  the  right  ol‘ (C.6)  goes  to  zero  as  c-»0. 
By  a  similar  analysis,  the  second  term  on  the  right  of  (C.6)  goes  to  zero  as  f— >0  and  therefore, 
(C.5)  holds. 
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Table  5:  Breakdown  poim  e  ,m  in  (15)  of  the  predictors  in 

G  "',m 
2 

(50)  and  for  the  nominal  process  2.  For  this  process, 
e  .  -r  ....  Vr,  Vm. 


m=4  m =5  I  m=6 


!  o.oooi 

j 

1.012 

mu 

;  i 

:  0.001  | 

1.054 

1.084 

• 

|  0.005 

1.139 

1.139 

i  o.oi 

1.203 

j  1 

1.181 

i 

#  !  0.025 

1.333 

1.271 

!  0.05 

1.482 

1.378 

i  0.075 

1  1.593 

1.461 

1.007  I  1.007 


1.291 

1.291 

1.370 

1.370 

BHBB 


0  10  I  1.678  |  1.52S  !  1.435  1.435  I  1.435  i  1.435 


j 

.583  I  1.491  1.4 


I  | 

1.799  1.629  I.53S  j  15-8  i  1.538  1.53 


!  0.20  !  1. 874  !  I.AOS  !  1.611  1.611  !  1.611  1.611 


I  0.15 


1 

.  0.30 


1  0.50 


0.70 


1.942  1.776  !  1.69S  I  1.698  1.698  1.698 


1.908  1.799  1.736  I  1.736  1.736  1.736 


1.786  .1734  1.691  1.691  1.691  |  1.691 


1 

! 

0.90 

1 

1  1.650 

1.636 

1.622 

1.622 

k  i 

i 

0.99 

! 

1.595 

1.596 

1.594 

1.594 

•»n 
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are  150  faculty  members,  a  majority  of  whom  conduct  research  in  addition  to  teaching. 

Research  is  a  vital  part  of  the  educational  program  and  interests  parallel  academic  specialties. 
These  range  from  the  classical  engineering  disciplines  of  Chemical,  Civil,  Electrical,  and  Mechanical 
and  Aerospace  to  newer,  more  specialized  fields  of  Biomedical  Engineering,  Systems  Engineering, 
Materials  Science,  Nuclear  Engineering  and  Engineering  Physics,  Applied  Mathematics  and  Computer 
Science.  Within  these  disciplines  there  are  well  equipped  laboratories  for  conducting  highly  specialized 
research.  All  departments  offer  the  doctorate;  Biomedical  and  Materials  Science  grant  only  graduate 
degrees.  In  addition,  courses  in  the  humanities  are  offered  within  the  School. 

The  University  of  Virginia  (which  includes  approximately  2,000  faculty  and  a  total  of  full-time 
student  enrollment  of  about  16,400),  also  offers  professional  degrees  under  the  schools  of  Architecture, 
Law,  Medicine,  Nursing,  Commerce,  Business  Administration,  and  Education.  In  addition,  the  College 
of  Arts  and  Sciences  houses  departments  of  Mathematics,  Physics,  Chemistry  and  others  relevant 
to  the  engineering  research  program.  The  School  of  Engineering  and  Applied  Science  is  an  integral 
part  of  this  University  community  which  provides  opportunities  for  interdisciplinary  work  in  pursuit 
of  the  basic  goals  of  education,  research,  and  public  service. 
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